


Vf Status from Applications Point of View 

• Applications to Real-World Problems 

- N-S solution of full configuration was a big goal in the 80s 

- Numerical procedures and computing hardware have been 
advanced enabling simulation of complex configurations 

• Some Examples of Successful Applications 

- Components of liquid rocket engine 

- Hydrodynamics (Submarines, propellers, ...) 

- Ground vehicles (automobile aerodynamics, internal flows...) 

- Biofluid problems (artificial heart, lung, ...) 

- Some Earth Science problems * 

• Current Challenges 

- For integrated systems analysis, computing requirement is very large 
=> Analysis part is still limited to low fidelity approach 

- For high-fidelity analysis, especially involving unsteady flow, long turn-around 
time is often a bottle neck => Acceleration of solution time is required 





A Major Drivers of Current Work 
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• To provide computational tools as an economical option for developing 
future space transportation systems (i.e. RLV subsystems development) 

Impact on component design => Rapid turn-around of high-fidelity analysis 
Increase durability /safety =* Accurate quantification of flow 

(i.e. prediction of low- induced vibration) 

Impact on system performance => More complete systems analysis 

using high-fidelity tools 

• Target 

Turbo- pump component analysis => Entire sub-systems simulation 
Computing requirement is large: 

=»The goal is to achieve 1000 times speed up over what was possible in 1992 






Objectives 
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• Mission of Conference 

To bring together Industry and Academia 4 (Government) 
To nurture the next generation in computational mechanics 

• Objective of the Current Talk 

To discuss some current issues in large scale computing for 
mission-oriented tasks 


Viscous Incompressible Flow 


• Formulation 

Can be viewed as a limiting case of compressible flow where the flow 
speed is insigr.if icant compered to the speed of sound (Preconditioned 
compressible N-S eq.) 

^Artificial compressibility approach 

- Artificial Compressibility Method (Chorin, 1967; Temam, 1977) 

ADI Scheme, FD (Central+diss) (BeanuS Warming, 1978; Briley-McDonald, 1977) 
LU-S6S, FV (Centrakdiss) (Yoon and Jameson, 1987...) 

Line Relaxation, FD (Upwind) („. MacCormack, 1985) 

GMRES, FD (Upwind) 

Or truly incompressible 

=* Pressure projection approach 

- MAC (Harlow and Welch, 1965) 

- Fractional Step Method (Chorin, 1968; Yanencko, 1971; Marchuck, 1975„._) 

- SIMPLE type Pressure Iteration (Caretto et al., 1972; Patanka 6 spalding, 1972...) 
=* Use derived variables 

- Vorticity- Velocity (Fasel, 1976; Dennis et al., 1979; Hafez et al., 1988 

- Stream function- vorticity 




Artificial Compressibility Method 
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• Formulation 

$ + ^ =0 

- Introduces hyperbolic behavior into pressure field. 

Speed of pressure wave depends on the artificial compressibility parameter, p, 

- The equations are to be marched in a time like fashion until the 
divergence of velocity converges to zero. 

=> Relaxes incompressibility requirement. 

Time variable during this process does not represent physical time step. 

For time-accurate solutions 

- Iterate the equations in pseudo-time level for each time step until 
incompressibility condition is satisf ied. 

Efficient sub-iteration is the key for success 





A f Pressure Projection Method(INS3D-FS) 
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• Approach in generalized coordinates 

- Finite volume discretization 

- Accurate treatment of geometric quantities 

- Dependent variables - pressure and volume fluxes 

- Implicit time integration 

- Fractional step procedure 

Solve auxiliary velocity field f irst, 

then enforce incompressibility condition by solving a Poisson 
equation for pressure. 

• Code performance 

- Computing time : 80 ms/grid point/iteration 

- Memory usage: 70 words/grid point 


Pressure Projection Method 


• Fractional-step 

- Solve for the auxiliary velocity field, using implicit predictor step: 

■ 7 - (U, - u ' ) = — Vp" + h(u ' ) 

A t 


- The velocity field at time level (n+1) is obtained by using a 
correction step: 

—(«’*' -u’) = -V/T’ + h(u ;" ) - Vp- + h{u] ) 

A t ' ‘ 

- The incompressibility condition is enforced by using a Poisson equation 
for pressure ( p' = />"' -p ) 

V-p' = —Vu 
At 








Current Challenges flU 
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• Challenges where improvements are needed 

- Time-integration scheme, convergence 

- Moving grid system, zonal connectivity 

- Parallel coding and scalability 

• As the computing resources changed to parallel and distributed 
platforms, computer science aspects become important such as 

- Scalability (algorithmic A implementation) 

- Portability, transparent coding etc 

• Computing resources 

- “Grid" computing will provide new computing resources for 
problem solving environment 

- High-fidelity flow analysis is likely to be performed using "super 
node" which is largely based on parallel architecture 


Supercomputing Memory and CPU Requirements 
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Parallel Implementation of INS3D 



INS3D-MLP (NAS MLP no pin-to-node) 
/ OpenMP 



TEST CASE : SSME Impeller 
60 zones / 19.2 Million points 
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Space Shuttle Main Engine Turbopump 
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Overset Grid System 




InJet Guide Vanes 


15 Blades 
23 Zones 
6.5 M Points 


Olffuw 
23 Blades 
31 Zones 
8.6 M Points 





Shuttle Upgrade SSME-rigl 
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grid for tip 

background grid 

clearance 

Imoeller Grid : 

60 Zones / 19.2 Million Grid Points 
Smallest zone 75K /Largest zone : 99 6K 

Less than 192 orphan points. 
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Summary of 55ME Turbopump Simulation 




Problem Size: 

- 34.3 Million Points 

- 800 physical time steps in one rotation 
CPU requirement: 

- One physical time-step requires less than 20 
minutes wall time with 80 CPU's on Origin 2000. 

- One complete rotation requires one- week wall 
time with 80 CPUs. 

I/O: 

- Currently I/O is through one processor. Timing 
will be improved with parallel I/O since time- 
accurate computations are I/O intensive 
Parallel Efficiency: 

MLP/OpenMP version requires 19-25% less 
computer time than MPI/OpenMP version. 

Pin- to- node for MLP version reduces computer 
time by 40% 


Discussion on Numerical Procedures 


• Finite Difference 

- Based on Taylor series expansion => Requires smooth grid 

- Meed specie! care fer grid singularity' 

- Generally easier to use fine grids near wail at high Reynolds number 

• Finite Volume 

- Formulation is more physical (conservation of properties) 

- Viscous flux calculation is not as straightforward 

- Difficult to implement higher order schemes 

• In actual implementation, however, 

- These differences become unclear 

i.e. - FV in curvilinear coordinates requires lots of averaging depending on 
definition of variables such as staggered vs cell vertex arrangement 
- Both FD and FV implementations are very similar near grid singularities 

- Major differences come from time integration scheme which also affects 
the computational efficiencies, especially, for unsteady flow computations 
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Parallel Implementation of INS3D 



INS3D-MLP (NAS MLP no pin-to-node) 
/OpenMP 
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• TEST CASE : SSME Impeller 
60 zones / 19.2 Million points 



Op*nMP Thread* per MLPGroup 


Number of CPU* 
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Parallel Implementation of INS3D 
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INS3D-MLP/OpenMP vs. -MPI/OpenMP 





TEST CASE : SSME Impeller 
60 zones / 19.2 Million points 






